[draft] Add native CDC support by tclinkenbeard-oai · Pull Request #13287 · apple/foundationdb

tclinkenbeard-oai · 2026-05-27T16:58:20Z

(This PR is a WIP)

This PR extends FDB to support natively streaming mutations to tracked key ranges. Clients register CDC streams and these are tracked via CDC proxies, which peek from the log system and pop once acknowledgements are received from clients. New pseudo-tags are added to mutations by commit proxies in order to support this. Overlapping CDC streams are supported, and a mutation can't be safely popped until all CDC streams have sent corresponding acknowledgements. This PR does not add support for the CDC interface in the bindings, that is planned for a future PR.

Before being marked ready for review, this feature will be knob-guarded and disabled by default

foundationdb-ci · 2026-05-27T17:02:07Z

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Commit ID: 9b4b2f5
Duration 0:03:36
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; if [[ $FDB_VERSION =~ 7\.\3. ]]; then echo skip; else exit 1; fi; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-27T17:02:53Z

Result of foundationdb-pr-clang on Linux RHEL 9

Commit ID: 9b4b2f5
Duration 0:04:20
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; if [[ $FDB_VERSION =~ 7\.\3. ]]; then echo skip; else exit 1; fi; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-27T17:02:58Z

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

Commit ID: 9b4b2f5
Duration 0:04:28
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; if [[ $FDB_VERSION =~ 7\.\3. ]]; then echo skip; else exit 1; fi; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2026-05-27T17:02:58Z

Result of foundationdb-pr-clang-ide on Linux RHEL 9

Commit ID: 9b4b2f5
Duration 0:04:24
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; if [[ $FDB_VERSION =~ 7\.\3. ]]; then echo skip; else exit 1; fi; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-28T16:04:42Z

Result of foundationdb-pr on Linux RHEL 9

Commit ID: 0761ac6
Duration 0:43:09
Result: ❌ FAILED
Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-28T16:11:04Z

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Commit ID: 0761ac6
Duration 0:49:32
Result: ❌ FAILED
Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-28T16:29:18Z

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

Commit ID: 0761ac6
Duration 1:07:47
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2026-05-28T21:57:33Z

Result of foundationdb-pr-macos on macOS Ventura 13.x

Commit ID: 2823578
Duration 0:20:28
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /usr/local/bin/bash --login ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-28T22:00:10Z

Result of foundationdb-pr-clang-ide on Linux RHEL 9

Commit ID: 2823578
Duration 0:23:06
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-28T22:01:51Z

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

Commit ID: 2823578
Duration 0:24:47
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2026-05-28T22:02:57Z

Result of foundationdb-pr on Linux RHEL 9

Commit ID: 2823578
Duration 0:25:49
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-28T22:08:16Z

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: 2823578
Duration 0:31:08
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /opt/homebrew/bin/bash --login ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-28T22:22:08Z

Result of foundationdb-pr-clang on Linux RHEL 9

Commit ID: 2823578
Duration 0:45:04
Result: ❌ FAILED
Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-28T22:24:18Z

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Commit ID: 2823578
Duration 0:47:10
Result: ❌ FAILED
Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

…re attrition

foundationdb-ci · 2026-05-28T23:29:40Z

Result of foundationdb-pr-macos on macOS Ventura 13.x

Commit ID: 93065de
Duration 0:17:40
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /usr/local/bin/bash --login ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-28T23:29:50Z

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: 93065de
Duration 0:17:48
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /opt/homebrew/bin/bash --login ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-28T23:36:32Z

Result of foundationdb-pr-clang-ide on Linux RHEL 9

Commit ID: 93065de
Duration 0:24:30
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-28T23:56:52Z

Result of foundationdb-pr on Linux RHEL 9

Commit ID: 93065de
Duration 0:44:48
Result: ❌ FAILED
Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-28T23:57:08Z

Result of foundationdb-pr-clang on Linux RHEL 9

Commit ID: 93065de
Duration 0:45:06
Result: ❌ FAILED
Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-28T23:59:26Z

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Commit ID: 93065de
Duration 0:47:24
Result: ❌ FAILED
Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-29T00:17:18Z

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

Commit ID: 93065de
Duration 1:05:14
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2026-05-29T15:00:50Z

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: 1bb10ee
Duration 0:17:16
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /opt/homebrew/bin/bash --login ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-29T15:02:31Z

Result of foundationdb-pr-macos on macOS Ventura 13.x

Commit ID: 1bb10ee
Duration 0:18:59
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /usr/local/bin/bash --login ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-29T15:05:26Z

Result of foundationdb-pr-clang-ide on Linux RHEL 9

Commit ID: 1bb10ee
Duration 0:21:52
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-29T15:26:37Z

Result of foundationdb-pr-clang on Linux RHEL 9

Commit ID: 1bb10ee
Duration 0:43:03
Result: ❌ FAILED
Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-29T15:27:32Z

Result of foundationdb-pr on Linux RHEL 9

Commit ID: 1bb10ee
Duration 0:43:56
Result: ❌ FAILED
Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-29T15:29:58Z

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Commit ID: 1bb10ee
Duration 0:46:22
Result: ❌ FAILED
Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-29T15:47:33Z

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

Commit ID: 1bb10ee
Duration 1:03:59
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

tclinkenbeard-oai added 26 commits May 27, 2026 05:57

Initial native CDC commit

d86ffe0

Add native CDC commit proxy routing

158333e

Add native CDC stream lifecycle metadata operations and tests

9ce0954

Fix correctness regressions

a28711d

Add native CDC acknowledgement watermarks

46e41a0

Add native CDC proxy consume and acknowledge backend

0bc5799

Recruit and publish CDC proxy endpoints through recovery routing

7203fe5

Persist and publish CDC stream ownership routing

9f52b0c

Add non-recovery CDC proxy failover and durable stream reassignment

9b76882

Preserve native CDC streams across recovery and proxy failover

230af4d

Add buffered native CDC consumption and safe shared-tag acknowledgements

92752c5

Add client-facing native CDC operations with proxy failover retries

4d48b1c

Finalize CDC retention state when streams are removed

e61abe0

Bound buffered mutation memory in CDC proxies

364ec1e

Avoid waiting forever during concurrent CDC stream removal

a8599a4

Bound CDC tags and persist stream ID allocation

1808150

Fix CDC proxy buffered byte release accounting

7acc707

Resume CDC buffers from the retained minimum version

dfb79ba

Refresh CDC buffer cursors after acknowledgements

f98c7f5

Anchor CDC retention at stream registration commit versions

7ada95e

Register the CDC shared-tag simulator test

d3c942d

Fix Native CDC recovery progress across log generations

eb58b1a

Contain Native CDC stream expiry without terminating its proxy

7a9e54e

Fix Native CDC workload consumption synchronization

f26afd0

Fix Native CDC recovery handling and stabilize simulator coverage

e0d06a4

Simplify serialize methods

9b4b2f5

tclinkenbeard-oai added 2 commits May 28, 2026 14:23

Fix native CDC recovery and replication across failure topologies

db90538

Add code probes for native CDC

2823578

tclinkenbeard-oai added 2 commits May 28, 2026 15:52

Fix native CDC proxy reassignment wakeups and initialize streams befo…

1dd8b80

…re attrition

Fix native CDC CI linkage and allow draining streams after disablement

93065de

Fix native CDC replay from satellite TLogs and add targeted coverage

185500c

Refine native CDC API boundaries and consumer coordination

1bb10ee

Conversation

tclinkenbeard-oai commented May 27, 2026

Uh oh!

foundationdb-ci commented May 27, 2026

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Uh oh!

foundationdb-ci commented May 27, 2026

Result of foundationdb-pr-clang on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 27, 2026

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 27, 2026

Result of foundationdb-pr-clang-ide on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 28, 2026

Result of foundationdb-pr on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 28, 2026

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Uh oh!

foundationdb-ci commented May 28, 2026

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 28, 2026

Result of foundationdb-pr-macos on macOS Ventura 13.x

Uh oh!

foundationdb-ci commented May 28, 2026

Result of foundationdb-pr-clang-ide on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 28, 2026

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 28, 2026

Result of foundationdb-pr on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 28, 2026

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Uh oh!

foundationdb-ci commented May 28, 2026

Result of foundationdb-pr-clang on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 28, 2026

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Uh oh!

foundationdb-ci commented May 28, 2026

Result of foundationdb-pr-macos on macOS Ventura 13.x

Uh oh!

foundationdb-ci commented May 28, 2026

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Uh oh!

foundationdb-ci commented May 28, 2026

Result of foundationdb-pr-clang-ide on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 28, 2026

Result of foundationdb-pr on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 28, 2026

Result of foundationdb-pr-clang on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 28, 2026

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Uh oh!

foundationdb-ci commented May 29, 2026

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 29, 2026

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Uh oh!

foundationdb-ci commented May 29, 2026

Result of foundationdb-pr-macos on macOS Ventura 13.x

Uh oh!

foundationdb-ci commented May 29, 2026

Result of foundationdb-pr-clang-ide on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 29, 2026

Result of foundationdb-pr-clang on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 29, 2026

Result of foundationdb-pr on Linux RHEL 9